functions (should probably be moved to another branch). Added density
and temperature profiles. Fixed LaTeX issues

regenerates from the saved NetCDF artifacts alone — no rerun, no raw
MS/ tree. Made list_diagnostics/load_series/load_hist_energy dispatch
between the MS/ HDF5 tree and a binary/ NetCDF dir, made field-energy
source-agnostic, and now persist HIST/energy.nc in save_run_datasets.
Added 3 tests

- regen harness: rebuild the full canned plot set from saved NetCDFs (no rerun)
- f(p) and delta-f lineouts; temperature profile from phase-space Maxwellian fits
- number-density profiles (initial/final/late-mean); 2-panel equal-aspect omega-k
- phase-space & spacetime: space on x-axis, cropped to box, log-contrast floor
- proper-LaTeX titles (prose vs math)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…ion+reflection plot (should probably be moved to osiris-lpi repo)

…space)

sync-up.sh script doesn't delete ongoing osiris runs

used

Given osiris.density.gradient_scale_length, BaseOsiris scales the simulation
box so the deck's linear density ramp realizes that gradient scale length at
the reference density (default n_c/4), mirroring adept's _lpse2d / kinetic_srs
grid sizing. The transform is a single spatial scale factor applied to
space.xmin/xmax, every profile.x, and every diag_species phase-space window;
grid.nx_p scales too, holding dx fixed (rounded up to a multiple of
node_number(1)). dt/tmax are untouched, so CFL is preserved. 1D decks only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Convert each diagnostic's MS/ HDF5 dumps into binary/*.nc as the run
produces them, instead of one monolithic read-all/write-once pass at job
end. This overlaps the (hours-long) PIC compute, reads each dump while
still warm in the node page cache rather than re-opening ~700k tiny files
cold off Lustre, and bounds conversion memory to a single dump instead of
the whole stacked (t, x) series. Addresses the I/O wall-time and
conversion-memory issues in osiris-lpi/postproc-performance.md (the
pcolormesh/fft2 plot-time OOM is independent and unaffected).

New adept/osiris/stream.py:
- StreamWriter: append-only (t, ...) NetCDF writer using an unlimited t
  dim grown one slot per dump, so the file is sized to exactly the dumps
  produced (no deck-derived size guess, no trailing-fill trim) and a
  restart resumes after the on-disk slots. Schema matches
  io.series_to_dataset, so load_series_nc / plots / regen are unchanged.
- convert_diagnostic_streaming: Stage A, memory-bounded conversion at
  job end.
- StreamConverter: Stage B, a best-effort daemon thread that drains
  completed dumps live (processing dump N only once N+1 exists, to avoid
  partial reads); a final sweep picks up the last dump. RAW diagnostics
  (variable particle count) stay on the batch concat path. Fully
  failure-isolated: never aborts the OSIRIS run, and the batch path
  remains the safety net.

Wiring: run_osiris spawns/finalizes the watcher around the existing
subprocess (finalize before error handling so it can't mask a failure);
base/post thread osiris.stream_convert + stream_poll_s through;
save_run_datasets reuses the watcher's files and stream-builds any it
missed. load_series_nc now flags a dim autoscaled only when its bounds
actually move (the streamer always records them).

On by default (osiris.stream_convert defaults true).

Tests: tests/test_osiris/test_stream.py covers Stage-A equivalence vs
load_series/batch, autoscale preservation, restart-resume, the N+1
hold-back rule, the live watcher thread, an end-to-end run_osiris run,
RAW-skip, and the reuse paths.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

OSIRIS has no asynchronous diagnostic I/O: every dump is a synchronous barrier
in the time loop, so on Lustre the ranks stall on each collective write
(~halving utilization at full dump cadence). `osiris.stage_root` works around
it as poor-man's async I/O: OSIRIS writes to a node-local /dev/shm ramdisk
(RAM-speed, no stall) and a background drainer mirrors each completed dump to
the durable run_root and reaps it from the ramdisk, so the RAM footprint stays
bounded by the poll interval rather than the whole run.

- stream.py: StreamConverter gains a `persist_dir` drain mode — mirrors grid
  AND raw dumps to persist_dir/MS, reaps the scratch, still streams grid .nc.
  persist_dir=None is byte-for-byte the old in-place behavior.
- runner.py: `stage_root` runs OSIRIS on the ramdisk, returns the durable dir
  as run_dir (post.py unchanged), syncs stragglers + reclaims the ramdisk at
  job end. Also ignore srun's spurious "couldn't chdir ... going to /tmp
  instead" launcher warning, which was false-positiving the exit-0 error guard.
- base.py: wire osiris.stage_root through.
- docs/osiris-adept-usage.md + tests.

Validated on Perlmutter (1024 ppc, nmin0.12, tmax=600): CPU 16-rank
250.7->207.0s (-17%), GPU 1xA100 128.8->123.7s (-4%); /dev/shm held bounded
while 3617 dumps drained to durable storage, full SRS post-processing intact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Add transverse_field_boundary_slabs(), a memory-bounded companion to
efield_lr_components for the boundary-light diagnostics (laser budget,
direction-split spectra/spectrogram) that only sample a thin slab at each
box edge. It loads each raw transverse field one at a time, slices it to the
two edge slabs, and frees the whole-grid array before the next — so the full
(t, x) left/right-going grid (tens of GiB per field on the long, wide SRS
runs) is never materialized. The Riemann split is local, so slice-then-combine
equals combine-then-slice (locked by test_boundary_slabs_match_full_split).

Also export render_cells / boxcar_downsample as public names so downstream
spectrograms can decimate to on-screen resolution before pcolormesh the way
plot_omega_k already does.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Prepares the general OSIRIS diagnostics for 2D decks that dump only
spatially-averaged fields plus Poynting lineouts (e.g. R-Follett):

- field_energy_components resolves -savg/-tavg field variants
  (_resolve_fld_diag), integrates over all spatial dims so 2D
  (t,x2,x1) series work, and emits per-component e1/e2/e3 energies.
- plot_epw_energy: the longitudinal (e1) field energy = electron
  plasma wave energy, added to save_canned_plots as
  epw_energy_vs_time.png (works for 1D and 2D).
- _total_field_energy sums only real field components
  (_is_field_component_dir), excluding s1 Poynting lineouts / slices
  that also live under FLD/; the per-dump reader falls back to the sole
  dataset when a savg dump names it differently.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The parser only recognized double-quoted strings, tracking string state
by toggling on `"` alone. Fortran single-quoted values (the common OSIRIS
convention, e.g. ext_fld='none') fell through _parse_atom's "keep as-is"
branch and were stored with their quotes, then double-wrapped on render
("'none'") — which OSIRIS rejects. Single quotes were likewise invisible
to _strip_comment, _split_top_commas, and the value scanner, so a `!`,
`,`, or `}` inside a single-quoted string (e.g. math_func expressions)
was mis-parsed.

Make all four string-state trackers handle single and double quotes
symmetrically via a shared _QUOTE_CHARS and an active-quote-char variable
(only the matching quote closes the string). The renderer already emits
double quotes, so single-quoted input now normalizes and round-trips
cleanly. Adds tests for single-quote parsing, render normalization, and
literal `!`/`,` inside single-quoted strings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

OSIRIS can exit non-zero yet have written usable data — most commonly a
segfault on MPI/CUDA teardown *after* "Simulation completed" (seen on
Perlmutter 2D CUDA runs), but also a mid-run death with dumps on disk.
Previously run_osiris hard-raised on any non-zero exit (or OSIRIS-reported
error), so __call__ aborted and post_process never ran — discarding a fully
completed run's diagnostics.

Now: on a non-zero exit / OSIRIS error, check whether the run produced output
(MS/ dumps, binary/ NetCDFs, or HIST/) via _run_produced_output(); if so, log
the error as a WARNING and fall through so the caller still consolidates
binaries and generates plots from the (possibly partial) data. Only a run that
produced nothing still raises. result now carries "crashed".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Multiple line/report diagnostics of the same field share one MS/ directory and
differ only by an index in the base name (e.g. two 's1,...,line' reports ->
s1-tavg-line-x2-01 / -02). _iter_from_name reads both -x2-0N-000000.h5 as
iteration 0, so keying a series by directory stacked them into one broken
NetCDF, losing the entrance/exit distinction the laser-transmission budget needs.

io: group a dir's dumps by base (_dump_base/_series_dumps); list_diagnostics
exposes each base as its own diagnostic via a synthetic <dir>/<base> handle;
_sort_dumps resolves that handle and selects one series (bare multi-series dir
raises). load_series / _diag_is_raw / convert_diagnostic_streaming compose
through _sort_dumps, so batch consolidation writes one NetCDF per series.
stream: the concurrent converter skips multi-report dirs (can't stream several
series as one) and leaves them to the batch pass. Single-series dirs unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…e h5

In ramdisk-staging mode the converter mirrored every completed grid dump
to the durable MS/ tree before deleting the scratch copy, so a staged run
still paid one inode + one copy per dump on the persist filesystem (~140k
h5 files per production SRS run). With osiris.stage_discard_h5: true the
grid dump is deleted after being appended to the streamed NetCDF and the
binary/*.nc become the only copy; RAW dumps are still mirrored (the batch
path builds their NetCDFs from the HDF5). Dumps whose stream failed are
left in place for the runner's final sync, so partial failures remain
recoverable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

list_diagnostics only walked MS/ whenever it existed, so under
stage_discard_h5 (MS/ holding only RAW) every grid diagnostic was
invisible to save_run_datasets and the plotting passes — the first
discard-mode run uploaded 2 of 14 binary artifacts and 2 of 53 plots.
Discovery now merges run_dir/binary NetCDFs for any diagnostic with no
raw dumps on disk, and the no-MS/ fallback prefers a run dir's binary/
subdir. final_iter falls back to scanning the whole MS/ tree (RAW dumps
carry iteration numbers) and field_energy_final is computed from the
last streamed FLD/*.nc slice when no FLD h5 exists. Regression tests for
the discard reap and the discovery/batch consolidation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

A full phase-space history decompresses to tens of GB in memory (x1gamma_q1
is ~48 GB: 11705 x 1000 x 1024; ~0.19 GB gzipped on disk, sparse ~250x), and
eager-loading one per plot OOM'd node bundles running many sims at once. Load
only what each plot needs:

- io.load_series / load_series_nc gain t_indices= (read just those time
  slices) plus a series_len() helper; a new open_series() context manager
  yields a lazily-backed array whose isel(t=it) reads one dump on demand, for
  consumers that walk the whole time axis reducing per dump.
- save_canned_plots' phase-space plots load the final dump + evolution panels
  via t_indices; plot_phasespace_evolution samples the same set in both call
  paths and takes its colour scale from the panels only; plot_profile gains
  avg_window= so a slice-selected series keeps the intended mean window.
- _decorate copies shallow (a deep copy materialized a lazily-opened series
  just to relabel its axes).

Docs: the usage guide documents the sliced/lazy loading convention for new
plotters.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMRKMZ3bRnUqLBjVtw7QJM

write_units now derives w_laser (rad/s), laser_wavelength (nm), laser_a0,
and laser_intensity — the peak intensity of a linearly polarized drive in
W/cm^2 (eps0 c E0^2/2 with E0 = a0 m_e c w_laser/e, i.e. the ICF convention
I * lam_um^2 = 1.37e18 * a0^2) — from the deck's antenna / zpulse_speckle /
zpulse section and the reference wp0. Laser-less decks are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The engine convention for _vlasov1d (and _vlasov2d, _pic1d) is the
RMS/standard-deviation thermal speed: Maxwellian exp(-v^2/2) at T=1,
L0 = lambda_De, code wavenumber k*lambda_De. normalization.py:91 was the
sole outlier with v0 = sqrt(2 T0/m_e), which made every logged physical
unit (v0, x0, c_light, box_length) wrong by sqrt(2), every dimensional
string input (um box sizes, gradient scale lengths, laser k0) off by
sqrt(2), and a 'normalizing_temperature: 2000eV' run physically a
4000 eV plasma. Dimensionless numeric-input dynamics were unaffected.

Also fixes a sign error in the NRL Coulomb logarithm
(log(n^0.5 / T^-1.25) == log(n^0.5 * T^+1.25)), which produced
logLambda_ee = -11.8 and a negative logged nuee at 2000 eV / 1.5e21cc;
the correct values are +7.22 and a positive rate.

vth_norm() is deliberately untouched: its only callers are in vfp1d,
which is self-consistently built on the sqrt(2T/m) convention.

Adds tests/test_vlasov1d/test_units_boundary.py: dimensional input in,
dimensional quantity out, checked against CODATA constants and the NRL
formulary - the units-boundary test class whose absence let this bug
survive every existing suite.

See VLASOV1D_CONVENTIONS_AUDIT.md (F1, F2) and
ADEPT_CONVENTIONS_AUDIT.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…in write_units

_tf1d re-implements the debye normalization inline; it carried the same
v0 = sqrt(2T/m) outlier (its engine, like _vlasov1d, runs in sqrt(T/m)
units) and the same log(n^0.5 / T^-1.25) sign error. Diagnostics-only:
the logged units were wrong; dynamics are unaffected (grid beta is
written but never consumed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…T0/mass

- Dougherty.compute_vbar returned the raw first moment n*u while its
  callers (find_self_consistent_beta, the drag term) treat it as a mean
  velocity. The drag therefore centered on n*u and momentum was not
  conserved wherever n(x) != 1. Now returns sum(v f)/sum(f).
- The Krook target Maxwellian was hard-coded to exp(-v^2/2) (T=1,
  m=1): any species with T0 != 1 was dragged toward T=1. It is now
  built with variance T0/mass from the species' bulk parameters, which
  are threaded through cfg.grid.species_params (new T0 entry).
- New test drives the collision operator in isolation with a 50%
  density modulation and a finite drift: per-cell density, momentum,
  and energy conservation, and the relaxed mean velocity must equal u0
  (the old operator drifts it toward n*u0).

Audit findings F4, F5.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…rgy monitor

- Field moments p and q were centered on the raw first moment n*u;
  they are now centered on the true mean velocity. The first moment is
  saved under the honest name 'j' and 'v' is now j/n (previously 'v'
  held the flux while the same integrand was called 'mean_j' in the
  scalars).
- The field-moment '-flogf' computed +int f log|f| dv, the exact
  negative of the scalar of the same name; both now agree.
- Adds mean_kinetic_energy / mean_field_energy / mean_total_energy
  scalars (electrostatic energy monitor, with the 1/2 factors). A
  conservation monitor of this kind would have caught the sqrt(2)
  convention bug far earlier.

Audit findings F6, F7, F12.2.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

dx was xmax/nx and the interpolation period was xmax; both are now the
box length (xmax - xmin). Latent for all shipped configs (xmin: 0.0)
but wrong spacing, kx grid, and advection wrap for any xmin != 0.

Audit finding F11.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

GridConfig.c_light (set only by wavepacket.yaml) was never read -
beta/c_light are always derived from the normalization. The
AmpereSolver class docstring claimed j = sum_s (q_s/m_s) int v f dv;
the code correctly has no 1/m_s.

Audit findings F12.1, F12.4.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The ey driver pair was a backscatter-matched triad under the old
c_hat = 11.30 (w_pump - w_seed = 1.16 = old EPW frequency) with
placeholder k0 = 1.0 on both drivers. Retuned to a matched triad under
the corrected c_hat = 15.984: pump (k0 = +0.162949, w0 = 2.79), seed
(k0 = -0.086080, w0 = 1.700942), EPW at k*lambda_De = 0.249. Matching
condition and conventions documented in the config.

Audit finding F3.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

- config.md gains a 'Normalization convention' section: v0 =
  sqrt(T0/m_e) (sigma convention), L0 = lambda_De, k in 1/lambda_De,
  Maxwellian exp(-v^2/2) at T=1, Bohm-Gross w^2 = 1 + 3k^2.
- Species v0/T0 documented as code-units-only (they bypass
  normalize()); drift and thermal width now share the same unit.
- FP/Krook 'baseline' documented as a rate in units of wp0 (no 2pi),
  with the newly logged nuee_norm as the reference scale, and the O(1)
  caveat between the Dougherty nu and the NRL nu_ee.
- Super-Gaussian alpha documented (code comment + config.md): it fixes
  <v^4>/<v^2> = 3 T0/mass for all m; the variance equals T0/mass only
  at m=2 (x1.24 at m=3, x1.37 at m=4). Documentation only, per review.

Audit findings F8, F9, F10 (nuee_norm logging itself landed with the
species_params change in an earlier commit).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The fixtures locked in the sqrt(2)-wrong units (c_light 11.302, v0
2.652e7 m/s, x0 12.14 nm, negative nuee). Regenerated under the
corrected normalization and the new config surface: c_light 15.984,
v0 1.876e7 m/s, x0 8.584 nm, logLambda_ee +7.22, positive nuee, new
nuee_norm, species_params T0, c_light knob removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Post-mortem of job 56005549 (srs-Ln100-Te4-5x-ions): at the every-3-step
field cadence the per-run drainer thread sustained only ~2/3 of the
~16 MB/s dump rate, so the /dev/shm staging backlog grew ~2.2 GiB/min,
filled the node's 256 GiB of RAM in ~1.8 h, and every later OSIRIS HDF5
write failed on ENOSPC (truncated 2 KB dumps, 2 OOM kills, 26 h TIMEOUT
at 16% of tmax). Details in osiris-lpi NOTES.md + dev_docs/
stream-drainer-recommendations.md.

Throughput (the drainer needed ~1.5x; this buys ~20x):
- StreamWriter batches appends and lands them chunk-aligned: one
  resize_dimension + one slice write per batch instead of a gzip
  read-modify-write of the whole ~1 MiB chunk per row (~40x write
  amplification). Measured 23x on the production field shape
  (1765 vs 76 appends/s; the job needed ~55/s per sim).
- Default compression gzip 4 -> 1 (the drainer is compression-bound;
  offline postproc can recompress).
- Discovery is cached: the full MS/ rglob walk, whose cost grows with
  the backlog, runs every rediscover_every polls instead of every poll.
- No thread pool on purpose: h5py serializes all HDF5 calls behind a
  process-global lock, so batching is where the throughput is.

Safety:
- Backlog spill valve (staging mode): past spill_backlog_files/_bytes,
  or under floor_free_bytes on the staging fs, a diagnostic flips to
  mirror-only draining (plain copy to persist MS/, ~10x cheaper), so
  the ramdisk keeps draining no matter what; the NetCDF is caught up
  from the mirror at finalize (mirror consumed + pruned in
  discard_grid_h5 mode). OSIRIS must never see ENOSPC on a dump.
- Corrupt dumps are quarantined (*.h5.bad) and the stream continues;
  previously one bad dump dropped the writer and the watcher re-hit the
  same file every poll, forever. Bookkeeping is now iteration-based
  (recovered from the iter coordinate on resume), not positional, so
  skipped dumps cannot shift slots.

Observability:
- One stats line per stats_every_s (streamed/spilled/backlog/
  quarantined + staging free space) so a smoke can assert the
  steady-state backlog is flat while OSIRIS runs; checking the ramdisk
  after the job proves nothing.
- Repeated identical errors log once per error_log_every occurrences.

API and on-disk schema unchanged (runner.py untouched); all 17 existing
stream tests pass unmodified, plus new coverage for quarantine and the
spill valve in both staging modes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>